Personalizing Embedded Assessment Sequences in Narrative-Centered Learning Environments: A Collaborative Filtering Approach

A key challenge posed by narrative-centered learning environments is dynamically tailoring story events to individual students. This paper investigates techniques for sequencing story-centric embedded assessments—a particular type of story event that simultaneously evaluates a student’s knowledge and advances an interactive narrative’s plot—in narrative-centered learning environments. We present an approach for personalizing embedded assessment sequences that is based on collaborative filtering. We examine personalized event sequencing in an edition of the CRYSTAL ISLAND narrativecentered learning environment for literacy education. Using data from a multiweek classroom study with 850 students, we compare two model-based collaborative filtering methods, including probabilistic principal component analysis (PPCA) and non-negative matrix factorization (NMF), to a memorybased baseline model, k-nearest neighbor. Results suggest that PPCA provides the most accurate predictions on average, but NMF provides a better balance between accuracy and run-time efficiency for predicting student performance on story-centric embedded assessment sequences.